Tensor sparsification via a bound on the spectral norm of random tensors

Given an order-d tensor A ∈ Rn×n×...×n, we present a simple, element-wise sparsification algorithm that zeroes out all sufficiently small elements of A, keeps all sufficiently large elements of A, and retains some of the remaining elements with probabilities proportional to the square of their magnitudes. We analyze the approximation accuracy of the proposed algorithm using a powerful inequality that we derive. This inequality bounds the spectral norm of a random tensor and is of independent interest. As a result, we obtain novel bounds for the tensor sparsification problem. As an added bonus, we obtain improved bounds for the matrix (d = 2) sparsification problem.